FIX #24374 - Data Contract at Data Product level#25314
Conversation
|
TypeScript types have been updated based on the JSON schema changes in the PR |
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
|
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
Co-authored-by: pmbrull <pmbrull@users.noreply.github.com>
|
gitar auto-apply:off |
🔍 CI failure analysis for 8651713: All CI failures are infrastructure-related (disk space exhaustion) or unrelated test failures. None are caused by this PR's Data Contract/Product inheritance changes.IssueLatest Maven SonarCloud CI Failure (Job 61195160379)Infrastructure Failure: Disk space exhaustion on GitHub Actions runner The runner ran out of disk space during the Maven build process, causing the job to fail. This is a GitHub Actions infrastructure issue, not a code problem. Python Integration Test Failure (Job 61186845538)Test Failure: The test expects PII classifications to be detected but receives an empty list. This is the same failure from previous runs. Test Statistics: 519 passed, 1 failed, 21 skipped (99.8% success rate) Previous Maven SonarCloud CI Failure (Job 61172493302)Test Failure: The test expects entity history version ordering to be descending but returns false, suggesting a version ordering issue. Test Statistics: 7900 out of 7901 tests passed (99.99% success rate) Previous Infrastructure Failures
Root CauseAll failures are completely unrelated to this PR's changes. This PR modifies Data Contract and Data Product inheritance functionality:
What this PR does NOT touch:
DetailsWhy the Latest Maven SonarCloud Failure Is Unrelated
Why Other Failures Are Unrelated
Evidence of Successful Implementation
Code Review 👍 Approved with suggestions 20 resolved / 24 findingsWell-structured implementation of Data Contract inheritance from Data Products. The schema changes, migrations, and backend logic are comprehensive with good test coverage. Previous findings remain as minor edge case considerations. More details 💡 4 suggestions ✅ 20 resolved💡 Edge Case: Semantic rules deduplication uses name only, ignoring rule content📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DataContractRepository.java In If an entity has a rule named "DataQuality" with one definition, and the data product has a rule named "DataQuality" with a different definition, the entity's rule wins entirely. This may be the intended behavior (as stated in the PR - "asset rules take precedence"), but there's no way to detect or warn about this conflict. Consider logging when a rule name collision is detected so administrators can understand why certain data product rules aren't being applied. 💡 Edge Case: Race condition when checking Data Product membership count📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DataContractRepository.java:655 The if (dataProducts.size() > 1) {
LOG.debug(
"Entity {} belongs to {} data products. Skipping contract inheritance to avoid ambiguity.",
entity.getId(),
dataProducts.size());
return entityContract;
}If an asset is being added to a second Data Product while another request is computing the effective contract, there could be a race condition where:
This is likely acceptable since the next request will see the correct state, but it could lead to brief inconsistency. Consider documenting this eventual consistency behavior in the API documentation. 💡 Performance: Multiple database lookups per asset in batch validationIn
When processing many assets (pagination is set to 100 per batch), this could result in significant database load. Consider:
This is acceptable for the scheduled app execution but could become problematic with large Data Products containing thousands of assets. 💡 Edge Case: Data Product contract status check may miss edge cases📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DataContractRepository.java:663 In if (dataProductContract != null
&& dataProductContract.getEntityStatus() != EntityStatus.APPROVED) {
dataProductContract = null;
}This is good defensive logic, but there's no logging or indication to users when inheritance doesn't happen due to non-approved status. Consider adding a debug log statement when a DP contract exists but isn't inherited due to its status. This would help with debugging when users expect inheritance but don't see it happening. Also, if the Data Product contract is in ✅ Quality: Inconsistent error message for inherited contract deletion
✅ Bug: Missing null check before setInherited calls in inheritFromDataProductContract
✅ Bug: validateContractWithEffective uses wrong entity reference for schema validation
✅ Performance: Unbounded asset retrieval in processDataProductContracts()
✅ Bug: Type casting circumvents TypeScript type safety
...and 15 more from earlier reviews What Works WellClean separation between entity contracts and inherited contracts with proper merge logic. Comprehensive schema migrations for both MySQL and PostgreSQL. Extensive test coverage including 854+ lines of backend tests and 1300+ lines of E2E Playwright tests covering inheritance scenarios. Tip Comment OptionsAuto-apply is off → Gitar will not commit updates to this branch. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
|



Describe your changes:
FIX #24374
Untitled.mov
I worked on ... because ...
Type of change:
Checklist:
Fixes <issue-number>: <short explanation>Summary by Gitar
termsOfUse,security, andslainherited only if empty in asset contractinheritedboolean flag toDataContract,termsOfUse,security,sla, andSemanticsRuletermsOfUsefrom string to object withcontentandinheritedfields via MySQL/PostgreSQL migrationsgetEffectiveDataContract()inDataContractRepositorycomputes inherited contractDataContractValidationApprefactored to validate inherited contracts across Data Product hierarchiesContractDetail,ContractSemantics,ContractSLADataContractInheritance.spec.ts(1287 lines) covering inheritance scenariosThis will update automatically on new commits.